AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models

Large language models(LLMs) exhibit excellent performance across a variety of tasks, but they come with significant computational and storage costs. Quantizing these models is an effective way to alleviate this issue. However, existing methods struggle to strike a balance between model accuracy and hardware efficiency. This is where we introduce AWEQ, a post-training method that requires no additional training overhead. AWEQ excels in both ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization. There is an observation that weight quantization is less challenging than activation quantization. AWEQ transfers the difficulty of activation quantization to weights using channel equalization, achieving a balance between the quantization difficulties of both, and thereby maximizing performance. We have further refined the equalization method to mitigate quantization bias error, ensuring the robustness of the model. Extensive experiments on popular models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing post-training quantization methods for large models

The vortex dynamics of a Ginzburg-Landau system under pinning effect

It is proved that the vortices are attracted by impurities or inhomogeities in the superconducting materials. The strong H^1-convergence for the corresponding Ginzburg-Landau system is also proved.Comment: 23page

Development of a New Risk Score for Incident Type 2 Diabetes Using Updated Diagnostic Criteria in Middle-Aged and Older Chinese

Type 2 diabetes mellitus (T2DM) reaches an epidemic proportion among adults in China. However, no simple score has been created for the prediction of T2DM incidence diagnosed by updated criteria with hemoglobin A1c (HbA1c) ≥6.5% included in Chinese. In a 6-year follow-up cohort in Beijing and Shanghai, China, we recruited a total of 2529 adults aged 50–70 years in 2005 and followed them up in 2011. Fasting plasma glucose (FPG), HbA1c, and C-reactive protein (CRP) were measured and incident diabetes was identified by the recently updated criteria. Of the 1912 participants without T2DM at baseline, 924 were identified as having T2DM at follow-up, and most of them (72.4%) were diagnosed using the HbA1c criterion. Baseline body mass index, FPG, HbA1c, CRP, hypertension, and female gender were all significantly associated with incident T2DM. Based upon these risk factors, a simple score was developed with an estimated area under the receiver operating characteristic curve of 0.714 (95% confidence interval: 0.691, 0.737), which performed better than most of existing risk score models developed for eastern Asian populations. This simple, newly constructed score of six parameters may be useful in predicting T2DM in middle-aged and older Chinese

Adaptive SPP–CNN–LSTM–ATT wind farm cluster short-term power prediction model based on transitional weather classification

With the expansion of the scale of wind power integration, the safe operation of the grid is challenged. At present, the research mainly focuses on the prediction of a single wind farm, lacking coordinated control of the cluster, and there is a large prediction error in transitional weather. In view of the above problems, this study proposes an adaptive wind farm cluster prediction model based on transitional weather classification, aiming to improve the prediction accuracy of the cluster under transitional weather conditions. First, the reference wind farm is selected, and then the improved snake algorithm is used to optimize the extreme gradient boosting tree (CBAMSO-XGB) to divide the transitional weather, and the sensitive meteorological factors under typical transitional weather conditions are optimized. A convolutional neural network (CNN) with a multi-layer spatial pyramid pooling (SPP) structure is utilized to extract variable dimensional features. Finally, the attention (ATT) mechanism is used to redistribute the weight of the long and short term memory (LSTM) network output to obtain the predicted value, and the cluster wind power prediction value is obtained by upscaling it. The results show that the classification accuracy of the CBAMSO-XGB algorithm in the transitional weather of the two test periods is 99.5833% and 95.4167%, respectively, which is higher than the snake optimization (SO) before the improvement and the other two algorithms; compared to the CNN–LSTM model, the mean absolute error (MAE) of the adaptive prediction model is decreased by approximately 42.49%–72.91% under various transitional weather conditions. The relative root mean square error (RMSE) of the cluster is lower than that of each reference wind farm and the prediction method without upscaling. The results show that the method proposed in this paper effectively improves the prediction accuracy of wind farm clusters during transitional weather